Managed notification system

ABSTRACT

A managed notification system compares image(s) and/or indicia relating to the image(s) and where there is a match selectively provides a notification of the same.

PRIORITY CLAIM AND INCORPORATION BY REFERENCE

This application is a continuation of U.S. patent application Ser. No.16/102,405 filed Aug. 13, 2018 which is a continuation of U.S. patentapplication Ser. No. 15/222,868 filed Jul. 28, 2016, now U.S. Pat. No.10,049,288 issued Aug. 14, 2018, which is a continuation of U.S. patentapplication Ser. No. 15/005,315 filed Jan. 25, 2016, now U.S. Pat. No.9,405,968 issued Aug. 2, 2016, which is a continuation-in-part of U.S.patent application Ser. No. 14/613,243, filed Feb. 3, 2015, now U.S.Pat. No. 9,245,190 issued Jan. 26, 2016, which is a continuation of U.S.patent application Ser. No. 12/177,103 filed Jul. 21, 2008, now U.S.Pat. No. 9,141,863 issued Sep. 22, 2015, all of the foregoing beingincorporated herein in their entireties and for all purposes.

U.S. Pat. Nos. (i) 5,561,718 issued Oct. 1, 1996, (ii) 6,038,337 issuedMar. 14, 2000, (iii) 8,254,699 issued Aug. 28, 2012 and (iv) 8,798,375issued Aug. 5, 2014 are incorporated herein in their entireties and forall purposes.

BACKGROUND Field of the Invention

The invention relates to a managed notification system. Moreparticularly, the invention relates to systems and methods ofrecognizing objects and/or making selective notifications influenced byrecognition results.

Discussion of the Related Art

Object recognition includes recognition of real-world objects ofinterest to a user. Where object recognition is recognition of aperson's face, biological indicators termed biometrics are typicallyinvolved.

SUMMARY

The present invention provides for matching one or multiple species anda managed notification dependent thereon. In an embodiment, a multiplematch notification system is for selectively notifying notificationrecipients, the system comprising: a video camera for capturing an imagein a video camera frame including primary captured content and asecondary captured content; cases stored in a digital library and acomputing device for accessing the library; in the library, pluralprimary content cases with respective primary content and pluralsecondary content cases with respective secondary content; in eachprimary content case, each primary content associated with a case groupselected from a list of multiple case groups; in each secondary contentcase, each secondary content associated with a label; a comparisonmodule of the computing device for identifying a primary match whenindicia derived from the primary captured content are matched withindicia corresponding to a library case; the comparison module of thecomputing device for identifying a secondary match when indiciacorresponding to the secondary captured content are matched with indiciacorresponding to a library case; wherein an image that results in bothprimary and secondary matches causes the system to test whether there isauthorization for the combination of the primary captured content andthe secondary captured content; and, wherein failure of theauthorization test results in a notification component varying with oneor both of the primary and secondary matched cases being sent tointended recipients identified via (i) links between case groups anduser groups such that one or more user groups intended to receive thenotification component can be identified and (ii) links betweenreceiving nodes and user groups so that one or more of plural receivingnodes that serve the intended recipients can be identified.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is described by way of example with reference to theaccompanying drawings wherein:

FIG. 1 is a block diagram of a managed notification system.

FIG. 2 is a block diagram of at least one image acquiring system of thenotification system, adapted to capture a first content.

FIG. 3 is a block diagram of the notification system including at leastone image acquiring system, a comparison module, and a notificationcomponent.

FIG. 4 is a block diagram of the notification system including thecomparison module, a content management module, and the at least onenotification component including at least one transmitted data point.

FIG. 5 is a detailed block diagram of the content management module.

FIG. 6 is a block diagram of at least one pre-selected receiving node.

FIG. 7 is a block diagram of an embodiment of the at least onepre-selected receiving node.

FIG. 8 is a block diagram of an embodiment of the notification system.

FIG. 9 is a block diagram of an embodiment of the notification system.

FIG. 10 is a block diagram of the notification system including an imageserver.

FIG. 11 is a block diagram of the notification system including a webserver.

FIG. 12 is a flow chart of a process of the comparison module.

FIG. 13 is a block diagram of an embodiment of the notification system.

FIG. 14 is a flow chart of a process of the at least one image acquiringsystem and comparison module.

FIG. 15 is a block diagram an embodiment of the notification systemproviding access to a third-party database.

FIG. 16 is a block diagram of a process of enrollment.

FIG. 17 is a block diagram of including an associated data to theprocess of enrollment.

FIG. 18 is a block diagram of the content management module activatingan event.

FIGS. 19-25 describe features and operation of selected embodiments of amultiple match notification system.

DETAILED DESCRIPTION OF THE INVENTION

A managed notification system is provided. What is disclosed herein caninclude, among other things, a user interactive feedback system which isover a communication medium such as the Internet. The techniquesdescribed below include a computer software application which, whenexecuted on the computer system of an user, configures that computersystem so that the user can receive and provide information to otherselected users or groups of users based on the result of a contentcomparison.

As used herein, the Internet refers at least to the worldwide collectionof networks and gateways that use the transmission controlprotocol/Internet protocol (TCP/IP) to communicate. The World Wide Web(WWW) refers to the total set of inter-linked hypertext documentsresiding on hypertext transport protocol (HTTP) servers, wherein webservers can be any type of web server, e.g. APACHE. As used herein, theWWW also refers at least to documents accessed on secure servers, suchas HTTP servers (HTTPS), which provide for encryption and transmissionthrough a secure port. As used herein, the term “web site” refers atleast to one or more related HTML documents and associated files,scripts, and databases that can be presented by an HTTP or HTTPS serveron the WWW. The term “web browser” refers at least to software that letsa user view HTML documents and access files and software related tothose documents.

It should be appreciated that any one or more elements of the systemillustrated in the following embodiments can be located remotely fromany or all of the other elements, and that any of the elements of agiven embodiment can, in fact, be part of another system altogether.Furthermore, it should be understood that aspects of the managednotification system enable one or both of object recognition and/orbiometric technologies, including but not limited to fingerprintrecognition systems, iris recognition systems, hand geometry systems,and signature recognition systems. At least some embodiments of theinvention are especially advantageous for biometric applications thatutilize information captured from an image.

FIG. 1 illustrates a managed notification system 100. The managednotification system 100 includes at least one image acquiring system 105which is adapted to capture a first content 110, a comparison module 140for comparing and distributing content within the system 100, a contentmanagement module 180 for controlling and managing the system 100, andat least one pre-selected receiving node 220.

FIG. 2 illustrates the at least one image acquiring system 105 in moredetail. The at least one image acquiring system 105 includes a lens 120,a processing unit 125, a digital sensor chip 130, a memory 135, and isadapted to capture the first content 110. The first content 110 iscaptured the by the at least one image acquiring system 105 andtransmitted to the comparison module 140 of the managed biometric-basednotification system 100.

FIG. 3 illustrates the comparison module 140 in more detail, and anotification component 170. The comparison module 140 is configured toreceive the first content 110, and includes at least one processor 145,at least one database 150, at least one search engine 160, and anassociated memory 165. The at least one search engine 160 is operativelycoupled with the at least one image acquiring system 105 and the atleast one database 150. The at least one processor 145 is operativelycoupled with the at least one image acquiring system 105 and the atleast one search engine 160, wherein the notification component 170 isgenerated provided a statistical threshold is satisfied between comparedcontent.

FIG. 4 illustrates the comparison module 140, the content managementmodule 180, the notification component 170, and at least one transmitteddata point 175. The notification component 170 is transmitted by thecomparison module 140 after the first content 110, and a second content155, located in at least one database 150 are compared. The contentmanagement module 180 is configured to manage and control thedistribution of the notification component 170 including the at leastone transmitted data point 175, to the user or users of the system 100.

FIG. 5 illustrates the content management module 180 in detail. Thecontent management module 180 provides the means for managing andcontrolling the managed notification system 100 functioning byinteracting with the comparison module 140. In providing the necessaryfunctionality to the user of the system 100, the content managementmodule 180 includes a case module (“CM”) 185, a notification module(“NM”) 190, a database browsing and reporting module (“DBRM”) 195, asubscription content module (“SCM”) 200, a support module (“SM”) 205,and an account management module 210 (“AMM”).

The content management module 180 serves as a graphical user interface(GUI), and provides the user interaction, feedback functionality, andthe ability to control and manage the managed biometric-basednotification system 100. In an embodiment, the content management module180 is accessed through a network via a web portal. The contentmanagement module 180 is embodied in a computer-readable executableprogram comprising computer executable instructions stored on computerreadable media, and is usable with a general purpose computer system. Inanother embodiment, the content management module 180 is embodied in atransmission medium, such as one or more carrier wave signalstransmitted between computers and a server, on a wireless network.

The CM 185 is a module configured to allow the user to manage and createcontent and information for a managed notification system such as anobject recognition or a biometric-based notification system 100. The CM185 provides the ability to create case information and associated caseidentification numbers (“CIN”), and also provide the user the ability tomodify case information. The user is able to create cases by“enrollment” of the first content 110 captured from the at least oneimage acquiring system 105. Enrollment is a process of collecting dataabout an object or individual and subsequently storing the data as in areference template representing identity. The procedure may be importantwhen a user is familiar with an object or individual who is nototherwise recognized by the system 100. The created content is thenstored in the at least one database 150 of the comparison module 140.

CM 185 provides the ability to also update the second content 155 withthe first content 110, provided the threshold or substantial similarityrequirement is met. This feature may be important where the managednotification system 100 has outdated information and current informationis available from the environment. The CM 185 is also configured toallow the user to group case information based on specific criteria. Forexample, a “case group” can be labeled “shoplifters,” very importantpersons,” or “persons with records.” The case group may be defined asany group where identification is important to the surroundingenvironment.

The CM 185 also manages stored content. The CM 185 generally containsfunctionality relating to an arrangement module within a database. Thefunctionality contains fields, which can also be used to search storedcontent (see DBRM 195), use menus, and tab functionality for arranginginformation. The contained fields can include object descriptors orrelationships, name, date of birth, sex, offense list, history,observations, transactions, and interactions in the environment.However, more fields can be added that may be specific to an individualor the environment, in order to better define the content and enhancesearch capabilities (see DBRM 195).

The NM 190 is a module configured to provide the user the means fornotifying at least one user of the managed notification system 100. TheNM 190, in conjunction with the comparison module 140, allows the userto select different methods of interacting with the users of the system100. The notification component 170 can be transmitted based on userpreferences or pre-selected input into the NM 190 of user(s) or usergroup(s) in which to receive the information. The user(s) or usergroup(s) defined in the NM 190 can be attached to single or multiplecase groups (configured by the CM 185). In an embodiment, the NM 190allows the user to determine which user(s) or user group(s) receive theat least one notification component(s) 170 and transmitted data point(s)175 based on time intervals, such as day, week, month and year.

The NM 190 also provides the user the ability to send a variety ofnotification components 170. In an embodiment, the notificationcomponent 170 can be transmitted from the comparison module 140, whichin an embodiment can include a notification server, based on differentpreferences or methods of delivery, e.g. SMS, MMS, Email, InstantMessaging, Voicemail, and the like. Furthermore, the NM 190 also allowsthe user to select several different types of content within the atleast one database 150 in which to be transmitted with the notificationcomponent 170.

In an embodiment, the user is also able to control the threshold inwhich the first and second content, 110 and 155, respectively, arecompared. However, this function can also be implemented within the AMM210. In an embodiment, the user is able to send a non-thresholdcomparison to an operator or automated operator module e.g.administrative authority, to determine substantial similarity, as aprimary or secondary measure in validating the comparison. This functioncan be utilized in cases where an “unconfirmed” or non-thresholdcomparison or comparisons result.

Numerous scenarios can be detailed in which specific information istransmitted to appropriate users at pre-selected times and methods, whoare selected to handle that type of information, or have requestedspecific information to be transmitted, based on a variety of factorsspecific to those individuals, as such, examples above are not designedto be limiting in any way.

The DBRM 195 is a module configured to provide the user the ability todevelop reports and search the at least one database 150 within themanaged notification system 100. The at least one database 150 issearchable by any case information filed within the system 100 throughthe fields created in the CM 185. The information can include records,notes, observations, details, and other information which can apprisethe user(s) as to the status or propensities of an individual in anenvironment. The user is able to access and view cases, generatereports, which include, but are not limited to, match, enrollment andnotification logs, as well as other internal statistics. The DBRM 195also provides the ability to export these logs and reports in anygeneral format.

The SCM 200 is a module configured to provide the user the ability tosubscribe, manage, and browse current and available databasesubscriptions. This module increases the referenced content, or secondcontent 155, within the managed biometric-based notification system 100.The system 100 also centralizes available databases and providessubscription information to the user, such as a description of theavailable database(s), associated pricing for each database, and areference to the overall content. The SCM 200 also provides the user theability to share databases, or form “Alliances,” with other subscriberswithin the system 100. However, this functionality, in an embodiment,can be included in the CM 185. The functionality of this module alsoprovides for the ability to request the authority who implements thesystem 100, to data mine other databases. The process of data mining,includes the ability to process data from other third party databases,and will create a database or databases based on processed or extractedfeature sets from facial images and other information found within thosedatabases.

The SM 205 is a module configured to provide customer interaction andsupport to the user of the content management module 180 with themanaged notification system 100 administrator, or authority, whoimplements the system 100. The SM 205 provides an interactive programconfigured to aid the user in resolving issues with the system 100. Theissues presented within the SM 205 can also be tracked and managed. TheSM 205 includes a frequently asked questions application providinganswers to commonly filed questions, and also includes other methods ofsupport including live help and chat programs.

The AMM 210 is a module configured to provide the administrator orauthority who implements the managed notification system 100, aninterface to manage user accounts, access, and data within the system100. Through the AMM 210, is the ability to set up preferences, as wellas rights to and within the system 100, including adding, deleting,modifying access for other users and sub-users of the CM 185. The AMM210 is also configured to add, edit, and remove the at least one imageacquiring system 105, the at least one pre-selected receiving node 220,as well as other components associated with the system 100. This featureis particularly important where more components are added due to anincrease in employees and/or areas in which to implement at least oneimage capturing device 105.

In an embodiment, the AMM 210 can be implemented with the comparisonmodule 140 by a single authority, having control over multipleimplemented managed notification systems 100. In this embodiment,remaining modules of the content management system 180 can beimplemented into an environment or be provided to the user through a webportal. In this embodiment, the users of the system 100 are able tointeract with the comparison module 140 through the modules describedherein. This functionality allows for control of implemented systems 100within plural environments and interaction between one or plural ones ofthose environments.

FIG. 6 illustrates the at least one pre-selected receiving node 220. Inan embodiment, the node is a terminal, which can be a plurality, with amotherboard on a computer network, including a central processor 230,associated memory 240 (e.g., DRAM, ROM, EPROM, EEPROM, SRAM, SDRAM, andFlash RAM) for storing programs and or data, an input/output controller250, optional special purpose logic devices (e.g., ASICs) orconfigurable logic devices (e.g., GAL and reprogrammable FPGA, a networkinterface 260, a display device 280, one or more input devices 290, ahard disk drive 300, a floppy disk drive 310, and a data bus 270 (e.g.,a SCSI bus, an Enhanced IDE bus, or a Ultra DMA bus) coupling thesecomponents, allowing communication therein.

The central processor 230 can be any type of microprocessor. The displaydevice 280 can be any type of display, such as a liquid crystal display(LCD), cathode ray tube display (CRT), or light emitting diode (LED),capable of displaying the generated outputs. The input device 290 can beany type-of device capable of providing the inputs described herein,such as keyboards, numeric keypads, touch screens, pointing devices,switches, styluses, and light pens.

The network interface 260 can be any type of a device, card, adapter, orconnector that provides the at least one pre-selected receiving node 220with network access to a computer or other networked enabled device. Inone embodiment, the network interface 260 enables the at least onepre-selected receiving node 220 to connect to a computer network such asthe Internet. The content management module 180, in an embodiment, isloaded into the pre-selected receiving node 220 via the hard disk drive300, the floppy disk drive 310, or the network interface 260, wherein inanother embodiment, is included within the comparison module 140.Alternatively, the program can reside in a permanent memory portion(e.g., a read-only-memory (ROM)) chip) of the main memory 240.

The at least one pre-selected receiving node 220 can be implemented onat least one of client and server. A “client” can be broadly construedto mean one who requests or receives a file, and “server” can be broadlyconstrued to be the entity that sends or forwards the file. In anembodiment, the at least one pre-selected receiving node 220 operates ina networked environment using a communication connection to connect toone or more remote computers. The remote computer can include a personalcomputer, server, router, network PC, a peer device, or other commonnetwork node. The communication connection can include a Local AreaNetwork (LAN), a Wide Area Network (WAN), or other networks.

FIG. 7 illustrates an embodiment of the at least one pre-selectedreceiving node 220. In an embodiment, the at least one pre-selectedreceiving node 220 is a transceiver or other similar means for receivingthe at least one notification component 170. The receiving means caninclude, but is not limited to, a cell phone, a personal digitalassistant (PDA) or other similar handheld portable device. Further, inthis embodiment the at least one pre-selected receiving node 220 canfunction as at least one image acquiring system 105, allowing the usermeans to capture the first content 110 and receive the notificationcomponent 170 and the at least one transmitted data point 175. The useof the transceiver supplies mobility and versatility to the user in theenvironment. The transceiver can be networked to other devices, such asin a client-server or peer to peer system, allowing the user to receiveand send information to another transceiver in the environment.

In use, upon implementation of the managed biometric-based notificationsystem 100 into an environment, the first content 110 is captured by theat least one image acquiring system 105. In an embodiment, the firstcontent 110 is an image of a person, wherein the image substantiallyincludes the facial region. In another embodiment, the first content 110is an electronic translation of handwritten or printed images intomachine text or optical character recognition (OCR), wherein the use ofmachine vision cameras or digital video cameras and computers is used toread and analyze alphanumeric characters. However, the first content 110can generally be any type of information important to the environment,and while only certain types of information is recognized by thecomparison module 140, the remainder of the information can becategorized, stored, and retrieved upon request by the user in the samemanner as compared content.

FIG. 8 illustrates, in one embodiment, the at least one image acquiringsystem 105 disposed in a commercial environment 320. In this embodiment,commercial establishments are able to implement the managed notificationsystem 100 to reduce the amount of “shrinkage” or other crimes occurringin the commercial environment 320. Illustrated in FIG. 9 , in anotherembodiment, stores within the same community or stores within the samechain, who are subscribers to the system 100, are able to share at leastone central database 325. The illustrated configuration will allow thesharing of information, via configuring the content management module180, pertinent to a certain store or stores, placing other “Alliances”on notice of a specific activity or person specific to the chain ofstores or community.

In an embodiment, the at least one image acquiring system 105 includes amachine vision camera. Machine cameras incorporate frames rates ofeither 15 fps (SXGA) or 34 fps (VGA), and include camera features suchas a standard processor, operating system, SD RAM and I/O connectivityvia Ethernet, USB serial port and monitor out. Components of a machinevision system will comprise several of the following: at least onedigital or analog camera with optics for acquiring images. An interfacefor digitizing images (widely known, in the case of CCTV cameras, as a“frame grabber”). A processor (often a PC or embedded processor, such asa DSP) (in some cases, all of the above are combined within a singledevice, called a smart camera). Input/Output hardware (e.g. digital I/O)or communication links (e.g. network connection) to report results.Lenses to focus the desired field of view onto the image sensor. Aprogram to process images and detect relevant features. In anembodiment, the machine vision camera includes a synchronizing sensorfor detection (often an optical or magnetic sensor) to trigger imageacquisition and processing. It can be understood by those skilled in theart that many available digital video and digital cameras can be used inconjunction with the managed notification system 100 as a means foracquiring images, as such, the examples are not limiting in any way.

In this embodiment, as illustrated in FIG. 10 , the at least one imageacquiring system 105, due to the large amounts of collected data, whichcan be uncompressed video, includes at least one server 385 on site tohandle the large quantities of data collection. In this embodiment, theat least one image acquiring system 105 transmits the first content 110to an image server 385, wherein the image server 385 would store thefirst content 110 and transmit to the comparison module 140 aftercomparison of the first and second content, 110 and 155, respectively,yields threshold level similarity (process illustrated in FIG. 14 ). Inanother embodiment, as illustrated in FIG. 11 , the at least one imageacquiring system 105 transmits the first content 110 to a web server 390of the comparison module 140. This configuration allows elements of thesystem 100 to be in various locations connected to a network.

After capturing the first content 110 from an environment, or otherwisecalled a “tracking environment,” the first content 110 is thentransmitted to the comparison module 140. This step is generally called“Image Acquisition.” Image Acquisition refers to how a computer getsimage data from a camera into the computer. In an embodiment, the atleast one image acquiring system 105 transmits uncompressed video to thecomparison module 140. In another embodiment, the at least one imageacquiring system 105 is on a network.

FIG. 12 illustrates the process of the comparison module 140. Thecomparison module 140 is configured to generate a set of descriptors orthe feature set 110 a from the received first content 110. Thecomparison module 140 includes the at least one search engine 160,operatively coupled with the at least one image acquiring system 105 andthe at least one database 150. The at least one search engine 160 isprogrammed to execute a series of instructions, stored on the memory165, to process a feature set 110 a of the first content 110 and asubstantially similar feature set 155 a of the second content 155.

If the feature sets 110 a and 155 a meet a predetermined threshold, thecomparison module 140 determines the comparison to be a match. Thenotification component 170 is then transmitted, and in an embodiment caninclude the notification server operatively coupled to the comparisonmodule 140. In another embodiment, the notification component 170 canalso be transmitted if a comparison does not result in a match. Asstated previously, the operator can be employed at this juncture todetermining substantial similarity. However, regardless of whether amatch occurs, the content is stored within the at least one database150. The notification component 170 can be comprised of any variety ofnotification made available by the NM 190, delivered at any preferencepresented by the same module. Furthermore, the at least one transmitteddata point 175 is also transmitted along with the notification component170, and its form and timing are determined by the same.

Illustrated in FIG. 13 , and in connection with FIGS. 10 and 11 , is anembodiment of the managed notification system 100. In this embodiment,the comparison module 140 receives the feature set 110 a of the firstcontent 110. However, the first content 110 remains stored on the imageserver 385 or web server 390, as illustrated in FIGS. 10 and 11 . In anembodiment, an extraction process is implemented within the localenvironment 320 at the at least one image acquiring system 105. In thisembodiment, as illustrated in FIG. 14 , the processor 125 of the atleast one image acquiring system is programmed to execute a series ofinstructions, stored on the memory 135, to process the feature set 110 aof the first content 110. Here, an algorithm or extraction module foundin the comparison module 140 is implemented at the at least one imageacquiring system 105. The comparison module 140 is then configured tocompare the feature set 110 a with the processed feature set 155 a ofthe second content 155, also illustrated in FIG. 10 . If the statisticalthreshold or substantial similarity is satisfied the first content 110is transmitted to the comparison module 140 upon request.

In an embodiment, the comparison module 140 includes a facialrecognition algorithm designed to process the feature set 110 a of thefirst content 110 and a feature set 155 a of the second content 155. Inanother embodiment, a process algorithm, or modules therein, areimplemented within the commercial environment at the at least one imageacquiring system 105 and processes the feature set 110 a, and comprisethe comparison module 140. The series of instructions used by thecomparison module 140 can include the “FACE-IT” facial recognitionsystem from Identix® Inc. (Merged with L-1 Identity Solutions Company)or “FaceVACS”® from Cognitec Systems. Generally, the widely availabletechnology can identify individuals independent of variances in humanfaces, including mimicry and aging variances, as well as variancescoming from a new hair style, glasses, or lighting changes.

The algorithms involved in comparing content are implemented todetermine the statistical closeness of the data involved, as such,multiple algorithms can be implemented for the determination of facialidentification, as well as algorithms identifying other features of anindividual in the environment, e.g. textual, to further limit thestatistical closeness of an individual. It can be understood by thoseskilled in the art that many available algorithms can be used inconjunction with the managed biometric based notification system 100 forcomparison of content, as such, the examples are not limiting in anyway.

As indicated above, in comparing the first and second content, 110 and155, respectively, the comparison module 140 draws from the at least onedatabase 150. However, the comparison module 140 in conjunction with thecontent management module 180 provides access to a variety of differentcontent forms, databases and servers. As FIG. 15 illustrates, in anembodiment, the database content can be predetermined, in that, it isthe result of subscriptions to public or third party databases, e.g.government, law enforcement agencies, state databases, or databases theuser requests to data mine. In another embodiment, as indicated above,the second content 155 is predetermined by the local environment 320,which can be based off internal company records and documents. As such,database content can be built from generated content within the system100 through user input and feedback. The second content 155 can also begenerated from outside the system 100 environment.

Illustrated in FIG. 16 is the enrollment process through the CM 185. Inthis embodiment, the user is able to enroll an individual into themanaged notification system 100. The system 100 comprehends the creationor enrollment when the feature set 110 a is processed and stored in theat least one database 155. As illustrated in FIG. 17 , the CM 185provides the ability to the user, to upload associated data 110 a withthe first content 110, e.g. notes on interactions and/or observationsrelated to the first content 110, as provided by the utility in FIG. 6 .

After the first content 110 and associated data 110 b are within the atleast one database 150, it becomes second content 155 and associateddata 155 b. In an embodiment, the comparison module 140 includesdatabases dedicated to the associated data 155 b, and in anotherembodiment, includes databases dedicated to each type of associated data155 b described herein. The second content 155 can then be manipulatedvia the content management module 180 by the user. The managednotification system 100 generally provides for the identification of anindividual already retained in the system 100 and image retention forthose individuals who are not within the system 100 but are to beidentified at a future date.

The degree in which the first and second content, 110 and 155,respectively, must match is deemed “substantial similarity.” This levelof similarity is a threshold or a predefined set of parameters set forthby the user of the managed notification system 100. These parameters maybe modified by users, and in an embodiment, users who have been givenauthority within the system 100, such as that defined by AMM 210. Thecomparison module 140 determines the substantial similarity of thefeature sets of 110 a and 155 a, and the notification component 170 andat least one transmitted data point 175 are transmitted as determined bythe content management module 180 if that similarity threshold is met.

In an embodiment, if the comparison module 140 transmits a plurality ofsecond content 155 corresponding to varying degrees of similarity amongcompared feature sets or where the feature sets of 110 a and 155 a donot meet the predetermined threshold of the system 100, the operatordetermines substantial similarity. And as mentioned above, the operatorcan also be implemented as a primary or secondary measure in determiningsubstantial similarity.

After a comparison is determined to provide a match as by meeting apredetermined threshold or by another means known to skilled artisans,the comparison module 140 transmits the notification component 170 basedon pre-selected criteria inputted into the content management module 180via the NM 190 by the user. The notification component 170 can becomprised of many different forms as set forth in the NM 190. Thenotification component 170 can be wirelessly communicated including butnot limited to utilizing short messaging service (SMS), multimediamessaging service (MMS), email, instant messaging or an Email to SMS,SMS to Email, MMS to Email, or Computer to SMS applications.

The at least one transmitted data point 175 is information transmittedalong with the notification component 170, and is comprised ofinformation which, in an embodiment, includes an image, e.g. thecomparison image of the comparison module 140 or any form of the secondcontent 155 described herein. In another embodiment, the at least onetransmitted data point 175 includes the associated data 155 b, or theassociated data 155 b along with the recently compared second content155. However, content can also be transmitted that describes theindividual's record or other persons that individual is known toassociate with, known accomplices, any other form mentioned herein.

Furthermore, in the criminal context, where a comparison of first andsecond content, 110 and 155, respectively, yields a substantiallysimilar threshold, the at least one notification component 170 is sentout detailing the identification of that criminal, and can includeassociated data 155 b, e.g. prior criminal history, bench warrant, stateprison, jail, conviction(s), arrest(s), charge(s), involved agencies,previous stolen items, treatment of staff, and gang affiliation. In anembodiment, the at least one transmitted data point 175 can includeaudio and/or textual information, e.g. instructions, guides, and/or usermanuals specific to a product in the environment, which can be recordedthrough the CM 185. However, the specifics of the at least onetransmitted data point 175 is dependent to large degree on the contentwhich is compared within the system 100. As such, general informationabout actions or events local to the commercial environment, or otherindividuals can also be transmitted. This feature can aid in providingcustomized service based on previous buying habits, amounts, andfrequently asked questions.

The notification component 170 serves more than one function in themanaged notification system 100. In another embodiment, the notificationcomponent 170 can activate or initiate an event. The comparison module140 transmits the notification component 170 as an instruction signal toan actuator 330 which functions to perform an event, e.g. motor, oractivate in application another networked device. In activating amotorized event, a servomechanism can be connected which supplies andtransmits a measured amount of energy for the operation of anothermechanism or system. In an embodiment, functional interfaces caninteract with an SMS gateway to perform functions in the environment.However, the notification component 170 can take a variety of forms inactivating events, as and as such, the form is not limited by theprovided examples.

FIG. 18 illustrates an embodiment, where a transducer or an actuatormodule 330 is included, which typically includes a clock 335, aprocessing unit 345, and a receiver 340, and is able to receive an inputsignal from the content management module 180 via the NM 190. Theactuator module 330 supplies as an output, a command via dry contact tocontrol a motor and the circuitry of a door e.g. gate controller or lockvia door strike or other similar device, in activating an event. In thisembodiment, control over the environment can occur via a controllerwhich can send an output command to a plurality of devices within anetwork. In another embodiment, a switch can be used to either turn onor off a device. In this embodiment, a binary command can be sent over anetwork to an interface, or remotely connect, to receivers controllingother devices and/or software in the environment.

Generally, the purpose of the at least one notification component 170 isto activate an event and/or notify the user of an individual. However,it can be understood by those skilled in the art that the notificationof an individual or activation of an event can take a variety of formsand activate a variety and plurality of users and devices, depending onthe context, as such, the examples are not limiting in any way.

In an embodiment, wherein the at least one notification component 170 isthen sent to a pre-selected receiving node 220, the receipt of the atleast one notification component 170 can occur on a stationary terminaland/or transceiver or other mobile device. In an embodiment, thepre-selected receiving node 220 is a plurality of transceivers. Theplurality of transceivers can transmit to other transceivers and/orretransmit information to other transceivers within the environment. Inthis embodiment, employees within a commercial environment 320 are ableto communicate and also send image data and enroll content in thecontent management module 180. In another embodiment, the transceivercan also serve as a “local” remote control, operating by sending awireless (e.g. infra-red or radio frequency) signal to a transceiverwhich is connected to a motor or application via another pre-selectedreceiving node 220 and a receiver. This embodiment provides the user ofthe system 100 another level of control in the environment.

The managed notification system 100 provides the ability to control andmanage information gathered in a real-time environment, and can interactwith that environment with up to date information from a variety ofexternal sources. The system 100 provides the user with an interactivesystem component which includes the ability to transmit information toavailable resources using a variety of technologies and providereal-time feedback in a dynamic environment.

An advantage of the managed notification system 100 is the contentmanagement module 180 element of the system 100. The content managementmodule 180 controls the distribution of content into an environment inwhich identification of information is crucial. The content managementmodule 180 allows the user to interact with the reminder of the system100 through, in an embodiment, a browser-based interface. The user isalso provided control over received, stored, and compared content withinthe system 100.

Specifically, an advantage of the managed notification system 100 is theability to manage the transmission of the at least one notificationcomponent 170. The content management module 180 allows the user theability to manage events in the environment, such as the activation ofapplications or actuators linked to other devices, and also users anduser groups in receiving information. This advantage intersects theresources in the environment with the result of the implementedalgorithm in a real-time scenario.

The system 100 also provides the user with the ability to manage thetype of notification, e.g. the mode of transmission. The time oftransmission is can also managed. Here, the notification component 170can be directed based a variety of predetermined factors e.g. time anddate intervals. While the transmission of at least one notificationcomponent 170 containing at least one data point 175 of an individualwho is known for shoplifting can be transmitted to the availablesecurity guards, there are many more uses directed to the system 100,such as, the purchase history of an individual transmitted to a salesassociate or a person who is known to have a heart condition to someoneon staff who understands Cardio Pulmonary Resuscitation.

The application also can be used in allowing specialized or certainpersons into a variety of areas in the environment. As such, it can beunderstood by those skilled in the art that many situations are possiblein an environment requiring information specific for that situation,which can be used in conjunction with the managed notification system100, therefore the above examples are not designed to be limiting inanyway.

Another advantage supplied by the content management module 180 is theability to allow the user the option of creating a variety of databasesand content within the managed notification system 100. The contentmanagement module 180 includes the ability to create databases bysubscription, data mining or create content from the local environment320. This option allows the user of the system 100 control over theorganization of the content.

The creation of content within the managed notification system 100 canalso include a variety additional data referencing created content. Assuch, an additional advantage of the system 100 is the ability toinclude associated data e.g. including images, notes, within the system100. The associated data can be uploaded by a variety of methodsidentical or equivalent to the at least one notification component 170.Furthermore, another advantage of the system 100 is the ability tomodify the stored content. Towards this end, the content managementmodule 180 provides security measures via the AMM 210, for those whomodify content within the system 100. Updated information about anindividual is important where an individual has had certain hostileinteractions with staff, maybe increasing in hostility, or is known tocarry a weapon, or has previously targeted a specific area of the store.Thus, if given authority, modification of content is crucial tomaintaining an updated system 100.

Another advantage of the managed notification system 100 is the at leastone pre-selected receiving node 220. In an embodiment, the at least onepreselected receiving node is a transceiver. The transceiver providesthe user the ability, not only to communicate crucial information to thenext user or staff member in the environment, and provide mobility, butalso activate applications or events based on that information. Thisplaces an added level of control to those within the environment.Moreover, while the transceiver is able to communicate to anothertransceiver in the environment, the transceiver is also able tocommunicate an image to the comparison module 140 and receive the atleast one notification component 170 based on the content transmitted.

The managed notification system 100 can be used in a variety ofenvironments. The system 100 can be implemented in the commercialenvironment 320, in perimeter surveillance, automated entry, customerrecognition or preauthorized visitor identification. In a retailenvironment, the system 100 will reduce operational costs and increasethe efficiency of the staff.

After being placed in strategic locations, the managed notificationsystem 100 will also reduce shrinkage. The system 100 accomplishes thisresult by increasing the probability in matching an individual with acomprehensive image in a database. The system 100 is constantly beingupdated and connected with content, coming from a variety of sources,e.g. the environment in which it is placed, third party databases,watch-lists, requested data mined sites. As such, retrieved informationis being compared against current data from a variety of sources,increasing the probability of a match.

The managed notification system 100 is designed to be employed in aplurality of locations and including a central station or headquarters.As such, first content 110 acquired from the plurality of locationsnetworked to the system 100 generally would be transmitted to thecentral station wherein the comparison module 140 would start theprocess illustrated in FIG. 10 , however, an advantage of the system 100is the implementation of the process within the at least one imageacquiring system 105 as illustrated in FIGS. 13 and 14 .

The advantage of processing the feature set 110 a at the at least oneimage acquiring system 105 is that it allows for only the feature set110 a to be transmitted for comparison. If the comparison achieves thedesired threshold of the system 100, the comparison module 140 requeststhe first content 100 from the at least one image acquiring system 105.This is particularly important where a large number of users of thesystem 100 are networked. This advantage reduces the amount of datatransmitted over a network, and thus reduces the amount of strain on theavailable bandwidth.

The descriptions above frequently refer to comparisons where a singleitem or species of interest (e.g. a facial image) is at issue, forexample where a facial image that is captured in a video image issomehow compared with a corresponding facial image or indicia thereofstored before the image capture takes place. These single matchcomparisons have utility in a number of situations such as identifying aperson using a facial image from the captured image. As indicated below,pairs of these or similar match comparisons involving multiple speciesalso have utility when they associate the same (e.g., first and secondcaptured facial images) or different (e.g., a captured and/or storedfacial image and a captured and/or stored inanimate object image)species.

In various embodiments, selected systems and methods described above canbe employed in whole or in part not only in single match applications,but also in multiple match applications. Multiple matches may includeany of multiple matches among similar or different species. Some suchmultiple match embodiments may be referred to as managed multiple matchselective notification systems.

The table of FIG. 19 shows several multiple match scenarios involvingtwo species, here one or both of faces and inanimate objects 1900. Forexample, a face might be that of a customer and the inanimate objectmight be a shoe such as a shoe worn by the customer. As shown in thetable, the species being matched correspond to either primary capturedcontent or secondary captured content. The primary and secondary contentmay be derived, captured, and/or previously stored images orinformation. The primary and secondary content may be derived from oneor multiple sources such as one or multiple images or frames capturedwith imaging equipment.

In particular, four scenarios are tabulated 1900 involving species pairs(1) face/face, (2) face/inanimate object, (3) inanimate object/face, and(4) inanimate object/inanimate object. In an exemplary use, a primarymatch involving primary content, in conjunction with a secondary matchinvolving secondary content, may be used in determining a result oroutcome that is a function of both the primary and secondary matchresults. And, in an exemplary use a primary match involving primarycontent may be used to determine, control, or influence a secondarymatching step or process.

FIG. 20 shows an image 2000 such as a video camera image. Included inthe image are species which may be isolated from the image and/orreferred to as primary captured content 2004 and secondary capturedcontent 2006. Although shown for clarity as non-overlapping content, theprimary and secondary captured content may overlap, for example wherethe primary captured content is a person's face and the secondarycaptured content is an earring worn by the person.

FIG. 21 shows isolation of species from an image 2100. As shown in theimage 2104, a man holds an upraised pistol. In a first scenarioconsistent with scenario 2 of FIG. 19 , primary captured content derivedfrom the image 2106 includes and/or isolates the man's face 2106 andsecondary captured content derived from the image includes and/orisolates the pistol 2108. Applicant notes this figure utilizes a sketchfor clarity of the illustration. Images subject to review for primaryand secondary content include images obtained from still shot, video,infrared, tomographic, isotopic/radiation, scintigraphic, magnetic, andother imaging devices known to skilled artisans. In an embodiment, animage from a live scene such as a commercial environment is captured bya video camera and the primary and secondary content are identified inthe image.

Skilled artisans will recognize that the database 150 in precedingfigures such as FIG. 1 provides a digital case library similar to thatof FIG. 22 . As such, the processor 145, memory 165, and search engine160 in preceding figures such as FIG. 1 provide means for processing thedata of digital case libraries such as the digital case library of FIG.22 .

In an embodiment, data processing means including a comparison modulesuch at the comparison module 140 of FIG. 1 provides a means foridentifying a primary match when indicia derived from the primarycaptured content 2004 are matched with indicia derived from orcorresponding to a library case (2210 or 2210 and 2250) and a means foridentifying a secondary match when indicia corresponding to thesecondary captured content 2206 are matched with indicia derived from orcorresponding to a library case (2250 or 2250 and 2210). As shown,primary content cases 2210 include primary content 2212 and a case groupassociation 2214. As shown, secondary content cases 2250 includesecondary content 2252 and a label association 2254.

FIG. 23 shows a first exemplary notification process 2300. In the firststep shown 2310, primary and secondary captured content is derived froma captured image and in the second step shown 2320 a comparison moduleperforms primary matching and secondary matching utilizing forcomparison primary and/or secondary content cases from a digital libraryof cases 2315.

A test authorization step 2330 may follow determination of a primarymatch and a secondary match in a preceding step 2320. Here, anauthorization database 2325 may be used to determine authorizedcombinations of one or more of (i) primary and secondary content and/orinformation associated with matched case content, (ii) primary andsecondary matches, (iii) primary and secondary captured content, (iv)cases and labels, and the like. For example, in the test authorizationstep, a determination may be made as to whether the combination of theprimary captured content and the secondary captured content ispermissible or authorized.

In a logic step 2340 following the test authorization step 2330, (i) afailed authorization or YES 2350 results in a step 2360 which sends anotification component to selected recipients while (ii) an unfailedauthorization or NO 2345 may return the process to the capture imagestep 2310.

For example where an image results in both primary and secondary matchesthe system tests 2330 whether there is authorization for the combinationof the primary captured content and the secondary captured content. Uponfailure of the authorization test 2350 a notification component varyingwith one or both of the primary and secondary matched cases is sent 2360to intended recipients.

Data links may enable notification component(s) to be sent to selectedand/or intended recipients. For example, FIG. 24 shows data links whichmay be used to notify selected and/or intended recipients 2400. Here,recipients are identified via (i) links between case groups and usergroups 2410 such that one or more user groups intended to receive thenotification component can be identified and (ii) links betweenreceiving nodes and user groups 2420 so that one or more of pluralreceiving nodes that serve the intended recipients 2430 can beidentified.

Notification components may include an image derived from the live scenethat includes the human face and the inanimate object and/or anidentification or description of one or both of the primary andsecondary captured content and/or one or both the primary and secondarymatched cases. In some embodiments, the system marks the derived imageas with color(s), pointer(s), cloud(s), or with other suitable means topoint out an inanimate object such as an item matched in the secondarymatch. See also the description of notification components made inconnection with FIGS. 1-18 above.

In an exemplary operation and use, primary captured content is matchedwith a person associated with a parolee case group and the secondarycaptured content is matched with an item associated with a weapon(s)label. Here, the authorization 2340 is failed 2350 and a notificationcomponent is sent to recipients determined at least in part by the casegroup associated with the case matched in the primary match. In variousembodiments, the system maintains an authorization database indicatingprohibited combinations of primary and secondary content and/orallowable combinations of primary and secondary content. While a casegroup+label (e.g., parolee+weapon) combination might be recorded asprohibited “positive combination,” a “negative combination” such as casegroup−label (eg., security guard−weapon) may similarly be recorded as aprohibited combination.

In various embodiments, the primary and secondary captured content maybe processed before matching. For example, the primary captured contentmay substantially isolate a person's face captured from a live sceneand/or the secondary captured content may substantially isolate aninanimate object carried by or within reach of the person. See e.g.,FIG. 21 .

In various embodiments, steps may be interposed between the stepsdescribed above, for example a step that derives indicia from primaryand secondary captured content following the capture image step 2310.For example, in a comparison to determine a match between primarycaptured content and the content in a case may include, steps mayinclude providing a comparable numerical representation of the content.For example, in a comparison to determine a match between primarycaptured content and the content in a case may include, steps mayinclude dividing the primary captured content into pieces, each piecebeing described by a numerical representation based at least in part onedge detection and comparing each such numerical representation with acorresponding or similarly derived numerical representation derived fromcontent in a case. See for example the description above, U.S. Pat. Nos.5,561,718 issued Oct. 1, 1996, 6,038,337 issued Mar. 14, 2000, 8,254,699issued Aug. 28, 2012 and 8,798,375 issued Aug. 5, 2014.

FIG. 25 shows a second exemplary notification process 2500. Similar tothe notification process of FIG. 23 , in the second notification processthe cases available for matching may be limited to less than all of thecases in the library 2315, to less than all of the primary contentcases, or to less than all of the secondary content cases.

In an embodiment, the cases available for matching in secondary matchingare limited by an optional case selector 2511. Here, the case selectoruses the case determined from primary matching and a link between casegroups and labels to determine which cases will be made available forthe secondary match.

For example, where a case determined from a primary matching isassociated with a parolee case group, links between (i) the parolee casegroup and (ii) labels gun, knife, and bat may serve to limit casesavailable for secondary matching to cases associated with the labelsgun, knife, and bat.

While certain exemplary embodiments have been described and shown in theaccompanying drawings, such embodiments are merely illustrative and notrestrictive of the current invention, and that this invention is notrestricted to the specific constructions and arrangements shown anddescribed since modification can occur to those ordinarily skilled inthe art.

The invention claimed is:
 1. A notification system comprising: pluralprimary content cases with respective primary indicia stored in adigital library; each primary content case associated with a case groupselected from multiple case groups; plural secondary content cases withrespective secondary indicia stored in the digital library; capturedcontent including one or more digital video camera frames with primarycaptured content and secondary captured content; a first comparisonwherein primary captured content is compared with primary indicia; afirst match exists if primary indicia of a primary content case andprimary captured content contain a matched face; if a first matchexists, a second match compares secondary captured content and secondaryindicia of a secondary content case; and, if a second match exists, anotification is sent to one or more persons indicated by the case groupassociated with the matched primary content case.
 2. The notificationsystem of claim 1 wherein the case groups indicate persons to benotified via links between the case groups and user groups such that oneor more persons in one or more of the user groups receive thenotification.
 3. The notification system of claim 1 wherein thesecondary match requires that the secondary indicia of the secondarycontent case and the secondary captured content contain a matched face.4. The notification system of claim 1 wherein the secondary matchrequires that the secondary indicia of the secondary content case andthe secondary captured content contain a matched inanimate object. 5.The notification system of claim 1 wherein the secondary match requiresthat the secondary indicia of the secondary content case and thesecondary captured content contain a matched weapon.
 6. The notificationsystem of claim 1 wherein the secondary match requires that thesecondary indicia of the secondary content case and the secondarycaptured content contain a matched firearm.
 7. The notification systemof claim 1 wherein the notification varies with one or both of theprimary match and the secondary match.
 8. The notification system ofclaim 1 wherein the notification varies with the case group.
 9. Thenotification system of claim 1 wherein the primary captured content andthe secondary captured content come from a common video frame.
 10. Thenotification system of claim 1 wherein the primary captured contentresults from the digital video camera frames observing a live scene.